High Efficiency Video Coding (HEVC) is a new coding standard that has been developed in recent years. In the High Efficiency Video Coding (HEVC) system, the fixed-size macroblock of H.264/AVC is replaced by a flexible block, named coding unit (CU). Pixels in the CU share the same coding parameters to improve coding efficiency. A CU may begin with a largest CU (LCU), which is also referred as coded tree unit (CTU) in HEVC. In addition to the concept of coding unit, the concept of prediction unit (PU) is also introduced in HEVC. Once the splitting of CU hierarchical tree is done, each leaf CU is further split into one or more prediction units (PUs) according to prediction type and PU partition. Several coding tools for screen content coding have been developed.
Unlike regular live video contents, video materials corresponding to screen contents often have integer displacement values. Therefore, conventional encoders normally interpret motion vectors in bitstreams as fractional pixel offsets, such as ¼ may unnecessarily increase the bitrate. On the other hand, fractional motion vector values are still very useful for contents corresponding to natural scenes such as camera-captured contents. Accordingly, adaptive motion vector resolution targeted to address the issue of different video contents has been described in JCTVC-50085 (Li, et al., “Adaptive motion vector resolution for screen content”, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11 19th Meeting: Strasbourg, FR, 17-24 Oct. 2014, Document: JCTVC-S0085).
According to JCTVC-50085, an adaptive MB resolution enable flag (i.e., adaptive_mv_resolution_enabled_flag) is signaled in SPS (sequence parameter set) to indicate whether adaptive motion vector resolution is applied as shown in Table 1. In the slice header, an integer MV flag (i.e., use_integer_mv_flag) is signaled (see note (1-2) in Table 1) to indicate whether a motion vector (MV) in the current slice uses integer pixel resolution (i.e., use_integer_mv_flag=1) or quarter pixel resolution (i.e., use_integer_mv_flag=0). As shown in Table 1, syntax element, use_integer_mv_flag is incorporated in the bitstream only when adaptive MV resolution is enabled as indicated by the-SPS level syntax element, adaptive_mv_resolution_enabled_flag (see note (1-1) in Table 1).
TABLE 1slice_segment_header( ) {Note   .....    if( adaptive_mv_resolution_enabled_flag )(1-1)    use_integer_mv_flag(1-2)   ....}
At the decoder side, MV is parsed and decoded in the same way regardless whether use_integer_mv_flag is 0 or 1 in the current slice. Before the interpolating process, MV is scaled based on use_integer_mv_flag as shown below according to JCTVC-S1005 (Joshi, et al., “High Efficiency Video Coding (HEVC) Screen Content Coding: Draft 2”, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11 19th Meeting: Strasbourg, FR, 17-24 Oct. 2014, Document: JCTVC-S1005).if use_integer_mv_flag==1,then mvLX<<=2,mvCLX<<=2.  (1)
In equation (1), X is equal to 0 or 1, mvLX represents the motion vector of the luma component associated with list LX (i.e, list L0 or L1), and mvcLX represents the motion vector of the chroma component associated with list LX (i.e, list L0 or L1). The operation ““mvLX<<2” means that mvLX is left-shifted by 2 and the result replaces the original mvLX. Similarly, the operation ““mvcLX<<2” means that mvLX is left-shifted by 2 and the result replaces the original mvcLX.
There are several problems in the current slice level MV adaptive resolution approach. First, when temporal motion vector prediction (TMVP) is applied and the use_integer_mv_flag in the collocated picture and in the current picture are different, the MV resolution of the motion vector predictor (MVP) and the MV in the current picture being predicted will be mismatched. In this disclosure, abbreviation MVP may also correspond to either motion vector prediction or motion vector predictor depending on the usage context.
FIG. 1 shows an example. In this scenario, use_integer_mv_flag in the current picture (110) is 0, use_integer_mv_flag in the collocated picture (120) is 1, and the collocated MV (124) is equal to (4, 4) for the reference block (122). According to the existing practice, the collocated MV (4,4) will be treated as a MVP for the current MV (114) of the current block (112) of the current picture (110) directly if TMVP is used. However, the collocated MV (4,4) in the collocated picture represents a motion vector value of (16, 16) when MV is expressed in the quarter-pixel resolution.
In another scenario, use_integer_mv_flag in the current picture is 1, use_integer_mv_flag in the collocated picture is 0, and the collocated MV is still equal to (4, 4). According to the existing practice, the collocated MV (4,4) will be treated as a MVP for the current picture directly if TMVP is used. However, the collocated MV (4,4) in the collocated picture represents a motion vector value of (1, 1) when the MV is expressed in the integer pixel resolution.
In both examples shown above, the resolution of the MVP does not match with the resolution of the current MV to be predicted. This will deteriorate the efficiency of TMVP.
Another problem arises due to the bit width limitation. According to JCTVC-S1005, the resulting values of mvL0 and mvL1 as specified above will always be in the range of 215 to 215−1, inclusive. With this limitation, it is guaranteed that mvL0 and mvL1 can be expressed in two bytes. However, under the condition that use_integer_mv_flag in the current picture is 1, use_integer_mv_flag in the collocated picture is 0 and the collocated MV is equal to (215−1, 215−1), mvLX will exceed the two byte limitation after the mvLX<<=2 and mycLX<<=2 operation if TMVP merging candidate is selected.
The adaptive MV resolution may also cause an issue in the deblocking process when MVs in the current slice have integer pixel resolution. In the deblocking process, boundary strength is determined to select corresponding deblocking filter for a block boundary. The boundary strength is determined according to various factors. Among the different factors, one factor is related to the motion vectors on both sides of a block boundary. In particular, |Q_MVx−P_MVx| and |Q_MVy−P_MVy| is compared to a threshold value of 4 to determine the filtering boundary strength, where Q_MV and P_MV represents motion vectors in two adjacent blocks, Q_MVx and Q_MVy represent the x and y components of Q_MV, and P_MVx and P_MVy represent the x and y components of P_MV.
The threshold value of 4 is designed with the assumption that MVs always use quarter-pixel resolution. However, when MVs in the current slice uses other MV resolution such as integer pixel resolution, the threshold value of 4 may not be appropriate.
IntraBC (Intra picture block copy) is another coding tool for HEVC screen content coding (HEVC-SCC) extension. For the coding units (CUs) using IntraBC mode, the prediction block is obtained from the reconstructed region of current frame. Then, the block vectors (BVs) and residual are coded. In the 20th JCT-VC meeting in Geneva, February 2015, it was agreed that the IntraBC is unified with Inter coding mode. That is, the current picture is treated as a reference picture and inserted into one or both reference picture lists. Block vector prediction and coding are the same as inter motion vector prediction and coding. This unification simplifies the codec design. However there are some remaining issues. For example, in current SCM the block vectors are always using integer resolution while motion vectors can be both integer resolution and quarter-pel resolution, switched at slice level. This may lead to a resolution mismatch during the deblocking stage and MV prediction stage.
In HEVC screen content coding (HEVC-SCC) extension, another coding tool, named adaptive color-space transform or in-loop color-space transform has been adopted. An example of decoding flow for the in-loop color-space transform is shown in FIG. 2. An additional module, i.e., inverse color-space transform (230) is included. Various modules are shown in FIG. 2 including entropy decoder (210), de-quantization (215), inverse transform (220), cross-component prediction (CCP, 225), motion compensation (235), Intra prediction (240), adder (245), deblocking filter (250), SAO (sample adaptive offset) filter (255) and DPB (decoded picture buffer, 260). The decoder also includes a first switch (S1) select inverse color-space transform (in the lower position) or bypass the inverse color-space transform (in the upper position). The decoder also includes a second switch (S2) select Inter prediction (in the upper position) or Intra prediction (in the lower position). Other than the inverse color-space transform (230), all other modules are standard decoder modules used in conventional HEVC. When a block is coded with the color-space transform enabled, the inverse color-space transform is invoked to convert the residual domain back to the original domain for the output from the conventional inverse DCT/DST transform and CCP. A flag is signaled to indicate the usage of color-space transform in a CU. For IntraBC (Intra picture block copy) and Inter modes, the flag is signaled only when there is at least one non-zero coefficient in the current CU. For Intra modes, the flag is signaled only when the chroma mode of the first PU (i.e., top-left PU within the CU) is coded with DM mode. DM mode corresponds to direct mode where Intra mode for the chroma component is the same as the Intra mode used for the luma component.
Two different color-space transforms are applied depending on whether the CU is coded in a lossless or lossy manner. The forward and the inverse color-space transforms for lossy coding use the YCoCg transform matrices, which are defined as follows:
                                          Forward            ⁢                                          :                            ⁢                                                          [                                                                                          C                      0                      ′                                                                                                                                  C                      1                      ′                                                                                                                                  C                      2                      ′                                                                                  ]                                =                                                    [                                                                            1                                                              2                                                              1                                                                                                  2                                                              0                                                                                      -                        2                                                                                                                                                -                        1                                                                                    2                                                                                      -                        1                                                                                            ]                            ⁡                              [                                                                                                    C                        0                                                                                                                                                C                        1                                                                                                                                                C                        2                                                                                            ]                                      /            4                          ,        and                            (        2        )                                                      Inverse            ⁢                                          :                            ⁢                                                          [                                                                                          C                      0                                                                                                                                  C                      1                                                                                                                                  C                      2                                                                                  ]                                =                                    [                                                                    1                                                        1                                                                              -                      1                                                                                                            1                                                        0                                                        1                                                                                        1                                                                              -                      1                                                                                                  -                      1                                                                                  ]                        ⁡                          [                                                                                          C                      0                      ′                                                                                                                                  C                      1                      ′                                                                                                                                  C                      2                      ′                                                                                  ]                                      ,                            (        3        )            wherein the original color space (C0, C1, C2) may correspond to (R, G, B), (Y, Cb, Cr) or (Y, U, V).
The forward color-space transform in lossy coding as shown in equation (2) is not normalized, which results in reduced signal when the transform is applied. Considering that the norm of the forward transform is roughly equal to √{square root over (6)}/4 for C0 and C2, and √{square root over (2)}/2 for C1, delta QPs (quantization parameters) of (−5, −3, −5) are used to compensate the reduced signal range for the three color components, respectively. In other words, when the color-space transform is applied, the quantization parameter is set to (QP−5, QP−3, QP−5) for the three components, respectively, where QP is the ‘normal’ QP value for the CU without the color-space transform. The QP adjustment to accommodate the signal range reduction color-space transform is performed for the quantization/de-quantization process. On the other hand, while the deblocking process also utilizes the QP values, only the normal QP values are used for the deblocking process.
In the specification of HEVC-SCC, the QP is adjusted as described in sub-clause 8.6.2. The quantization parameter qP is derived as follows:
If cIdx is equal to 0,qP=Qp′Y+(cu_residual_act_flag[xTbY][yTbY]?−5:0)  (4)
Otherwise, if cIdx is equal to 1,qP=Qp′Cb+(cu_residual_act_flag[xTbY][yTbY]?−5:0)  (5)Otherwise (cIdx is equal to 2),qP=Qp′Cr+(cu_residual_act_flag[xTbY][yTbY]?3:0)  (6)where cu_residual_act_flag[xTbY][yTbY] is 1 if adaptive color-space transform is applied in the block with left-top position (xTbY, yTbY). Otherwise, cu_residual_act_flag[xTbY][yTbY] is 0. Qp′Y, Qp′Cb and Qp′Cr correspond to original quantization parameters for color component Y, Cb and Cr respectively. The expression (cu_residual_act_flag[xTbY][yTbY]?−5:0) will return a value (−5) if cu_residual_act_flag[xTbY][yTbY] is true or equal to 1, and will return 0 if cu_residual_act_flag[xTbY][yTbY] is false or equal to 0. The variable cIdx mentioned above corresponds to color index.
However, the scaling process controlled by QP is defined in sub-clause 8.6.3 of the HEVC standard according to:d[x][y]=Clip3(coeffMin,coeffMax,((TransCoeffLevel[xTbY][yTbY][cIdx][x][y]*m[x][y]*levelScale[qP%6]<<(qP/6))+(1<<(bdShift−1)))>>bdShift).  (7)
For k=0.5, the list levelScale[ ] is specified as levelScale[k]={40, 45, 51, 57, 64, 72}.
The qP adjustment in equations (4), (5) and (6) may cause a qP to be less than 0 when the adaptive color-space transform is applied. When qP<0, “qP %6” will result in a negative argument for the the list levelScale[ ] and lead to an undefined levelScale[ ] value. Therefore, d[x][y] is undefined in equation (7). Therefore, it is desirable to overcome this issue.